Range camera

ABSTRACT

An active illumination range camera that acquires a range and a picture image of a scene and provides a reflectance for a feature in the scene responsive to a distance for the feature provided by the range image, a registered irradiance for the feature provided by the picture image and registered irradiance for a region of a calibration surface corresponding to the feature provided by an image of the calibration surface acquired by the range camera.

BACKGROUND

A range camera acquires distances to features in a scene that the rangecamera images and provides the distances in a range image. Typically,the range camera is an active lighting range camera and comprises alight source that the camera controls to transmit light to illuminatethe scene. The range camera images light that the features reflect fromthe transmitted light back to the camera on pixels of a photosensor thatthe camera comprises to provide data which may be processed to determinethe distances and generate the range image. In a time of flight rangecamera the camera provides the range image by determining how long ittakes the transmitted light that is reflected and imaged on the pixelsto make a round trip from the camera to the features and back to thecamera. The camera uses the round trip times and the speed of light todetermine the distances. In a stereo range camera, the camera providesthe range image by determining binocular disparity for features in thescene responsive to the transmitted light that is reflected and imagedon the pixels. The camera triangulates the features responsive to theirdisparity to determine the distances. Usually, the transmitted light isinfrared light (IR). In addition to a range image of a scene, a rangecamera may also provide a contrast image, a “picture image”, of thescene responsive to intensity of the transmitted light that is reflectedby the features of the scene and reaches the camera. The picture imageprovides recognizable images of features in the scene and isadvantageous for identifying the features for which the range imageprovides distances. The picture image and range image of a scene may beacquired by imaging the scene on a same photosensor or on differentphotosensors registered to each other.

SUMMARY

An aspect of an embodiment of the disclosure relates to providing arange camera that provides a reflectance image of a scene for which therange camera acquires a range image and a picture image. The reflectanceimage provides a reflectance for features in the scene imaged on pixelsof the range camera photosensor for which the range camera providesdistances.

To provide the reflectance for the features in the scene, the camera iscalibrated in accordance with the disclosure by imaging under knowncalibration imaging conditions a “calibration surface” having knownreflectance on a photosensor of the range camera to acquire a“calibration image”. The calibration image comprises measures of amountsof light that pixels in the photosensor register responsive toirradiation by light from regions of the calibration surface imaged onthe pixels under the calibration conditions. An amount of light that apixel registers from a region of the calibration surface may be referredto as registered calibration irradiance (RCI). A reflectance of a givenfeature of the scene imaged on a pixel of the range camera photosensorin the picture image is determined responsive to an amount of light,hereinafter also referred to as registered scene irradiance (RSI),registered by the pixel for the given feature in the picture image,distance of the feature from the camera determined from a range image ofthe scene, and the RCI of a corresponding pixel in the calibrationimage. The corresponding pixel is a pixel that images a region of thecalibration surface corresponding to the given feature. Thecorresponding region of the calibration surface is a region that therange camera illuminates to acquire the calibration image along a samedirection that the range camera illuminates the given feature of thescene to acquire the picture image of the scene. The RCI for acorresponding pixel on which the corresponding calibration region isimaged in the calibration image is used to normalize the RSI to providea reflectance for the region of the scene and a reflectance image of thescene.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF FIGURES

Non-limiting examples of embodiments of the disclosure are describedbelow with reference to figures attached hereto that are listedfollowing this paragraph. Identical features that appear in more thanone figure are generally labeled with a same label in all the figures inwhich they appear. A label labeling an icon representing a given featureof an embodiment of the disclosure in a figure may be used to referencethe given feature. Dimensions of features shown in the figures arechosen for convenience and clarity of presentation and are notnecessarily shown to scale.

FIG. 1 schematically shows calibrating a range camera in accordance withan embodiment of the disclosure;

FIG. 2A schematically shows the range camera shown in FIG. 1 determiningreflectance for a region of a scene perpendicular to an optic axis ofthe range camera, in accordance with an embodiment of the disclosure;

FIG. 2B schematically shows the range camera shown in FIG. 1 determiningreflectance for a region of a scene that is not perpendicular to theoptical axis in accordance with an embodiment of the disclosure; and

FIG. 3 schematically shows a human computer interface comprising a rangecamera operating to determine reflectance of a user's facial skin inaccordance with an embodiment of the disclosure.

DETAILED DESCRIPTION

In the following detailed description a method of operating a rangecamera to acquire a calibration image of an optionally planarcalibration surface in accordance with an embodiment of the disclosureis discussed with reference to FIG. 1. A method of using the calibrationimage to determine reflectance of a region of a scene imaged by therange camera in accordance with an embodiment of the disclosure isdescribed with reference to FIG. 2A. In FIG. 2A the region of the scenefor which reflectance is determined is, by way of example, parallel tothe calibration surface shown in FIG. 1. How the method may be modifiedto determine reflectance for a region of the scene tilted with respectto the calibration surface, in accordance with an embodiment of thedisclosure, is discussed with reference to FIG. 2B. FIG. 3 schematicallyshows a human computer interface comprising a range camera that operatesto determine a user's facial reflectance in accordance with anembodiment of the disclosure. In the discussion, characters in boldscript represent a vector. The characters in normal script represent theabsolute value of the vector. A character in bold script accented by acap represents a unit vector.

In the discussion, unless otherwise stated, adjectives such as“substantially” and “about” modifying a condition or relationshipcharacteristic of a feature or features of an embodiment of thedisclosure, are understood to mean that the condition or characteristicis defined to within tolerances that are acceptable for operation of theembodiment for an application for which it is intended. Unless otherwiseindicated, the word “or” in the description and claims is considered tobe the inclusive “or” rather than the exclusive or, and indicates atleast one of, or any combination of items it conjoins.

It is noted that methods of providing a range camera with a calibrationimage and using the calibration image to determine reflectivity offeatures of a scene imaged by the camera in accordance with anembodiment of the disclosure are substantially independent of cameradesign and how the range camera determines distance to the features.Different range cameras calibrated and operated in accordance with anembodiment of the disclosure may therefore be expected to provide samereflectance images for a same scene substantially independent of how therange cameras operate to acquire range and picture images of the scene.The reflectance images may also be expected to be substantiallyindependent of imaging conditions, such as, for a same spectrum,intensity of illumination of the scene and distances of the cameras fromthe scene.

As a result, image processing algorithms that process images of scenesacquired by a range camera calibrated in accordance with an embodimentof the disclosure to provide functionalities that may be responsive toreflectance of features in the scenes may be generic to range cameratypes. That is, the image processing algorithms may process areflectance image of a scene to provide functionalities substantially ina same way and provide substantially a same result independent of typeand design of a range camera that provides the image and/or imagingconditions under which the range camera acquired the image. Amongfunctionalities responsive to reflectance of features that the imageprocessing algorithms may advantageously provide include by way ofexample, person identification, gesture recognition, determination ofemotional or physiological state from changes in skin tone, and avatargeneration.

FIG. 1 schematically shows a range camera 20 imaging a calibrationsurface 50 under calibration imaging conditions to acquire a calibrationimage for use in determining reflectance of features in a scene that thecamera images, in accordance with an embodiment of the disclosure. Rangecamera 20 is shown very simplified and may operate in accordance withany of various methods for determining distances to features in a scenethat the range camera images and providing a range image of the sceneresponsive to the determined distances. Range camera 20 optionallydetermines distances to the features by time of flight and/ortriangulation.

Optionally, range camera 20 comprises a light source 30 controllable totransmit, optionally infrared (IR) light to illuminate features in ascene the camera images, a photosensor 22 having pixels 23, and anoptical system, which is represented by, and referred to as, a lens 24.Lens 24 collects light reflected by the features from light transmittedby light source 30 and images the reflected light onto pixels 23 ofphotosensor 22. The lens has an optical axis 25, optical center 26 and adiameter “D” that optionally defines an optical aperture, α=πD²/4, ofthe lens and the optical system the lens represents. A unit vector{circumflex over (γ)} lies along, and represents a direction of opticalaxis 25. Photosensor 22 is substantially perpendicular to optical axis25 and located at a focal distance, “f”, from optical center 26. A pixel23 comprised in photosensor 22 is located in the photosensor by pixelcoordinates measured long x and y axes shown along edges of thephotosensor. A pixel coordinate origin from which pixel coordinates maybe measured is located at an intersection of optical axis 25 andphotosensor 22. A pixel located at pixel coordinates x_(i) and y_(j) maybe represented by p(x_(i),y_(j)) or p(i,j).

A controller 28 controls operation of range camera 20. The controllermay by way of example, control intensity and modulation of light thatlight source 30 transmits to illuminate features in an imaged scene andshuttering of the camera to provide desired exposure periods for imagingtransmitted light that is reflected by the features back to the rangecamera on pixels 23. Controller 28 may process light reflected by thefeatures and registered by pixels 23 during the exposure periods todetermine distances to the features and therefrom a range image of thescene and/or a picture image of the scene.

A pixel in a range camera, such as a pixel 23 in photosensor 22,registers incident light that irradiates the pixel by accumulatingpositive or negative electric charge, also referred to as “photocharge”,provided by electron-hole pairs generated by photons in the incidentlight. Circuitry in the range camera may convert photocharge accumulatedby the pixels into voltages that are used as measures of the amounts ofphotocharge they respectively accumulate and thereby of irradiance ofthe pixels. An amount of light that a pixel registers, also referred toas registered irradiance, may refer to an amount of optical energy thatirradiates the pixel, an amount of photocharge accumulated by a pixelresponsive to irradiance, or to any representation of the accumulatedphotocharge such as a voltage, current, or digital data generatedresponsive to the accumulated photocharge.

Spatial locations of features in a field of view (FOV) of range camera20 relative to the range camera are defined by spatial coordinatesmeasured along X, Y, and Z axes of an optionally Cartesian coordinatesystem 41. The Z axis is optionally coincident with optical axis 25 ofrange camera 20. Light source 30 is located along the X axis at adistance X_(S) from optical center 26 and has an illumination axis 32,which is optionally parallel to optical axis 25 and intersects the Xaxis at a distance X_(S). Spatial locations of features in the FOV ofrange camera 20 relative to light source 30 may be defined by spatialcoordinates measured along “primed” axes X′, Y′, and Z′ of a primedcoordinate system 42. Optionally, the Z′ axis is coincident withillumination axis 32, and the X′ and Y′ axes are respectively parallelto the X and Y axes. Origins of coordinate systems 41 and 42 may beconsidered to be, and may be referred to as, the locations of lens 24and light source 30.

Calibration surface 50 is optionally planar, and has a known reflectance“ρ_(O)”, which, optionally, is the same for substantially all regions ofthe calibration surface. The calibration conditions under which rangecamera 20 images calibration surface 50 optionally comprise a knowndistance and orientation of calibration surface 50 relative to the rangecamera optical axis 25, known range camera settings, such as apertureand exposure period, and a known optical output power with which lightsource 30 illuminates the calibration surface. By way of example, inFIG. 1 calibration surface 50 is perpendicular to optical axis 25, andis located at a distance Z_(O) from optical center 26 along the Z axis.

Let a region of calibration surface 50 imaged on a given pixel p(i,j,)shown in FIG. 1 be represented by

(X_(i),Y_(j),Z_(O)), where X_(i) and Y_(j) are X and Y coordinates ofregion

(X_(i),Y_(j),Z_(O)) in coordinate system 41. Coordinates X_(i) and Y_(j)correspond to coordinates x_(i), y_(j) of pixel p(i,j) in photosensor 22and lie along a line that extends from substantially a central point inregion

(X_(i),Y_(j),Z_(O)), passes through optic center 26 and endssubstantially at a central point in pixel p(i,j). Coordinatex_(i)=−(f/Z_(O))X_(i) and coordinate y_(j)=−(f/Z_(O))Y_(j). Pixel p(i,j)on which

(X_(i),y_(j),Z_(O)) is imaged may be represented by p(i,j,Z_(O)) toassociate the pixel with

(X_(i),Y_(j),Z_(O)), and to indicate that the feature, region

(X_(i),Y_(j),Z_(O)), imaged on the pixel is located at Z coordinateZ_(O).

(X_(i),Y_(j),Z_(O)) is located at a distance defined by a distancevector, “R(i,j,Z_(O))” that extends from optical center 26 tocoordinates X_(i),Y_(j),Z_(O) at a zenith angle θ(i,j,Z_(O)) relative tothe Z axis of coordinate system 41 and at an azimuth angle φ(i,j,Z_(O))relative to the X axis of the coordinate system. The absolute value of|R(i,j,Z_(O))|≡R(i,j,Z_(O))=[X_(i) ²+Y_(j) ²+Z_(O) ²]^(1/2). In primedcoordinate system 42, region

(X_(i),Y_(j),Z_(O)) has coordinates X′_(i)=(X_(i)−X_(S)), Y′_(j)=Y_(j),and Z′=Z_(O), and is located at distance defined by a distance vectorR′(i,j,Z_(O)) directed from light source 30 at a zenith angleθ′(i,j,Z_(O)) relative to the Z′ axis and at an azimuth angleφ′(i,j,Z_(O)) relative to the X′ axis.|R′(i,j,Z_(O))|≡R′(i,j,Z_(O))=[X′_(i) ²+Y′_(j) ²+Z′_(O)²]=^(1/2)=[X_(i)−X_(S))²+Y_(j) ²+Z_(O) ²]^(1/2). Arrows along distancevectors R(i,j,Z_(O)) and R′(i,j,Z_(O)) indicate direction of propagationof light from light source 30 to region

(X_(i),Y_(j),Z_(O)) and therefrom to pixel p(i,j) of photosensor 22. Aunit normal to

(X_(i),Y_(j),Z_(O)) is represented by {circumflex over (η)}(i,j,Z_(O)).If “A_(p)” represents the area of a pixel p(i,j), and

(i,j,Z_(O)) the area of region

(X_(i),Y_(j),Z_(O)), then,

(i,j,Z _(O))=A _(p)(Z _(O) /f)².  (1)

If, under calibration conditions, optical power per unit solid angle oflight that light source 30 transmits in a direction defined by a givenzenith angle θ′ and azimuth angle φ′ is represented by a vectorP_(O)(θ′,φ′), then optical power per unit solid angle that light source30 transmits in the direction of

(X_(i),Y_(j),Z_(O)) is P_(O)(θ′(i,j,Z_(O)),φ′(i,j,Z_(O))), which, forconvenience, may also be written P_(O)(θ′,φ′,(i,j,Z_(O))). A totalamount of optical power, hereinafter also referred to as “totalirradiance” (TIR), incident on

(X_(i),Y_(j),Z_(O)) from light source 30 under calibration conditions istherefore equal to the area

(i,j,Z_(O)) times the scalar product of {circumflex over (η)}(i,j,Z_(O))and P_(O)(θ′,φ′,(i,j,Z_(O))) divided by the square of the distance fromlight source 30 to

(X_(i),Y_(j),Z_(O)). In symbols, letting TIR(

(X_(i),Y_(j),Z_(O))) represent a total irradiance of region

(X_(i),Y_(j),Z_(O)) by light source 30 under calibration conditions,TIR(

(X_(i),Y_(j),Z_(O))) may therefore be written:

TIR(

(X _(i) ,Y _(j) ,Z _(O)))=

(i,j,Z _(O)){circumflex over (η)}(i,j,Z _(O))·P _(O)(θ′,φ′,(i,j,Z_(O)))/R′(i,j,Z _(O))²,  (2)

where “·” indicates a scalar product.

Since calibration plane 50 is assumed parallel to photosensor 22,{circumflex over (η)}(i,j,Z_(O)) is parallel to axis Z and axis Z′ and,

{circumflex over (η)}(i,j,Z _(O))·P _(O)(θ′,φ′,(i,j,Z _(O)))=P_(O)(θ′,φ′,(i,j,Z _(O)))cos θ′(i,j,Z _(O))=P _(O)(θ′,φ′,(i,j,Z _(O)))[Z_(O) /R′(i,j,Z _(O))].  (3)

The expression for TIR(

(X_(i),Y_(j),Z_(O))) in equation (2) may therefore be written,

TIR(

(X _(i) ,Y _(j) ,Z _(O)))=

(i,j,Z _(O))P _(O)(θ′,φ′,(i,j,Z _(O)))[Z _(O) /R′(i,j,Z _(O))³].  (4)

Substituting the expression for

(i,j,Z_(O)) from equation (1) provides

$\begin{matrix}\begin{matrix}\left. {{\left. {{{{{{TIR}\left( {\left( {X_{i},Y_{j},Z_{o}} \right)} \right)} = {{A_{p}\left( {Z_{o}/f} \right)}^{2}{P_{o}\left( \theta ’ \right.}}},\phi}’},\left( {i,j,Z_{o}} \right)} \right)\left\lbrack {Z_{o}/R}’ \right.}\left( {i,j,Z_{o}} \right)^{3}} \right\rbrack \\{\left. {{\left. {{{{= {\left( {A_{p}/f^{2}} \right){P_{o}\left( \theta ’ \right.}}},\phi}’}\left( {i,j,Z_{o}} \right)} \right)\left\lbrack {Z_{o}/R}’ \right.}\left( {i,j,Z_{o}} \right)} \right\rbrack^{3}.}\end{matrix} & (5)\end{matrix}$

Assuming that

(i,j,Z_(O)) is a diffuse reflector that reflects light with equaloptical power per unit solid angle in all directions in a hemispherecentered on

(i,j,Z_(O)) and facing range camera 20,

(i,j,Z_(O)) reflects light towards lens 24 along a direction ofR(i,j,Z_(O)) defined by zenith and azimuth angles θ(i,j,Z_(O)) andφ(i,j,Z_(O)), with “reflected” optical power per solid angle,

RP _(O)(θ,φ,(i,j,Z _(O)))={circumflex over (R)}(i,j,Z _(O))ρ_(O)TIR(

(X _(i) ,Y _(j) ,Z _(O)))/2π,  (6)

where {circumflex over (R)}(i,j,Z_(O)) is a unit vector alongR(i,j,Z_(O)) in a direction from

(i,j,Z_(O)) to lens 24.

Reflected optical power incident on aperture α of lens 24 from region

(X_(i),Y_(j),Z_(O)), that is total irradiance, TIR(α,

(X_(i),Y_(j),Z_(O))), of lens 24 by reflected light from

(X_(i),Y_(j),Z_(O)) may therefore be given by an expression:

$\begin{matrix}\begin{matrix}{{{TIR}\left( {\alpha,{\left( {X_{i},Y_{j},Z_{o}} \right)}} \right)} = {\left( {\alpha/{R\left( {i,j,Z_{o}} \right)}^{2}} \right){\hat{\gamma} \cdot {{RP}_{o}\left( {\theta,\phi,\left( {i,j,Z_{o}} \right)} \right)}}}} \\{= {\left( {\alpha/{R\left( {i,j,Z_{o}} \right)}^{2}} \right){\hat{\gamma} \cdot}}} \\{{{{\hat{R}\left( {i,j,Z_{o}} \right)}\rho_{o}{{{TIR}\left( {\left( {X_{i},Y_{j},Z_{o}} \right)} \right)}/2}\pi},}}\end{matrix} & (7)\end{matrix}$

where (α/R(i,j)²){circumflex over (γ)}·{circumflex over (R)}(i,j,Z_(O))is a solid angle subtended by aperture α at

(X_(i),Y_(j),Z_(O)). Noting that {circumflex over (γ)}·{circumflex over(R)}(i,j,Z_(O))=cos θ(i,j,Z_(O))=Z_(O)/R(i,j,Z_(O)), expression (7) maybe rewritten:

TIR(α,

(X _(i) ,Y _(j) ,Z _(O)))=(αZ _(O) /R(i,j,Z _(O))³)ρ_(O)TIR(

(X _(i) ,Y _(j) ,Z _(O)))/2π.  (8)

Using the expression for TIR(

(X_(i),Y_(j),Z_(O))) given by equation (5), and rearranging terms,equation (8) becomes,

TIR(α,

(X _(i) ,Y _(j) ,Z _(O)))=ρ_(O)(A _(p)α/2πf ²)Z _(O) ⁴ P_(O)(θ′,φ′,(i,j,Z _(O)))[R(i,j,Z _(O))R′(i,j,Z _(O))]⁻³.  (9)

Let τ, represent a coefficient that relates optical power, TIR(α,

(X_(i),Y_(j),Z_(O))), incident on pixel p(i,j) to the registeredcalibration irradiance RCI(i,j,Z_(O)), that the pixel provides as ameasure of total irradiance TIR(α,

(X_(i),Y_(j),Z_(O))). RCI(i,j,Z_(O)), may be given by the equation:

$\begin{matrix}\begin{matrix}{{{RCI}\left( {i,j,Z_{o}} \right)} = {\tau \; {{TIR}\left( {\alpha,{\left( {X_{i},Y_{j},Z_{o}} \right)}} \right)}}} \\\left. {{{{= {\tau \; {\rho_{o}\left( {A_{p}\; {\alpha/2}\pi \; f^{2}} \right)}Z_{o}^{4}{P_{o}\left( \theta ’ \right.}}},\phi}’}\left( {i,j,Z_{o}} \right)} \right) \\{\left. {\left\lbrack {{R\left( {i,j,Z_{o}} \right)}R}’ \right.\left( {i,j,Z_{o}} \right)} \right\rbrack^{- 3}.}\end{matrix} & (10)\end{matrix}$

Measurements RCI(i,j,Z_(O)) acquired for a plurality of pixelsp(i,j,Z_(O)) that image different regions

(X_(i),Y_(j),Z_(O)) of calibration surface 50 provide a calibrationimage for range camera 20. The calibration image may be used todetermine reflectances of features in a scene imaged in a picture imageof the scene acquired by range camera 20, in accordance with anembodiment of the disclosure as described below.

FIG. 2A schematically shows range camera 20 imaging a feature, a regionA(X_(i),Y_(j),Z_(k)) of a scene (not shown), to acquire a picture imageof the scene and determine reflectance of the region.A(X_(i),Y_(j),Z_(k)) is located at coordinates X_(i),Y_(j),Z_(k) ofcoordinate system 41, has an unknown reflectance ρ(i,j,k) and has a unitnormal {circumflex over (η)}(i,j,Z_(k)), which may be written{circumflex over (η)}(i,j,k), parallel to optic axis 25. RegionA(X_(i),Y_(j),Z_(k)) has an area A(i,j,Z_(k))=A_(p)(Z_(k)/f)², islocated at a distance defined by a distance vector R(i,j,Z_(k)) thatextends from light source 30 at a zenith angle θ(i,j,k) relative to theZ axis and at an azimuth angle φ(i,j,k) relative to the X axis.|R(i,j,Z_(k))|≡R(i,j,Z_(k))=[X_(i) ²+Y_(i) ²+Z_(k) ²]^(1/2). RegionA(X_(i),Y_(j),Z_(k)), is imaged on a pixel p(i,j,k) of photosensor 22.With respect to coordinate system 42, region A(X_(i),Y_(j),Z_(k)) islocated at coordinates X′_(i),=(X_(i)−X_(S)), Y′_(j)=Y_(j), and Z′=Z_(k)determined by a distance vector R′(i,j,Z_(k)) extending from lightsource 30 at zenith and azimuth angles θ′(i,j,k) and φ′ (i,j,k)respectively. |R′(i,j,Z_(k))|≡R′(i,j,Z_(k))=[X′_(i) ²=Y′_(j) ²+Z′_(k)²]^(1/2)=[X_(i)−X_(S))_(i) ²+Y_(j) ²+Z_(k) ²]^(1/2). Arrows alongdistance vectors R(i,j,Z_(k)) and R′(i,j,Z_(k)) schematically indicatepropagation of light from light source 30 to pixel p(i,j,k). It is notedthat when range camera 20 images the scene comprising regionA(X_(i),Y_(j),Z_(k)), calibration surface 50 is not present and is shownin FIG. 2A to show geometrical relationships between regionA(X_(i),Y_(j),Z_(k)) and regions of the calibration surface.

Let an amount of light, that is, registered scene irradiance RSI, thatpixel p(i,j,k) registers for light from region A(X_(i),Y_(j),Z_(k)) whenacquiring the picture image of the scene comprising A(X_(i),Y_(j),Z_(k))be represented by “RSI(i,j,Z_(k))”. Assume that controller 28 operateslight source 30 to illuminate the scene with optical powerP_(O)(θ′,φ′,(i,j,Z_(O))). Then, similar to determining RCI(i,j,Z_(O)) inaccordance with equation (10) for pixel p(i,j,Z_(O)) that images region

(X_(i),Y_(j),Z_(O)) of calibration surface 50 under calibrationconditions, RSI(i,j,Z_(k)), may be determined by an equation:

RSI(i,j,Z _(k))=τρ(i,j,k)(A _(p)α/2πf ²)Z _(k) ⁴ P _(O)(θ′,φ′,(i,j,Z_(k)))[R(i,j,Z _(k))R′(i,j,Z _(k))]⁻³.  (11)

To determine ρ(i,j,k) for A(X_(i),Y_(j),Z_(k)) according to anembodiment of the disclosure, controller 28 controls range camera 20 toacquire a range image for the scene comprising regionA(X_(i),Y_(j),Z_(k)) to determine a distance, R(i,j,Z_(k)), from therange camera to region A(X_(i),Y_(j),Z_(k)). The controller usesdistance R(i,j,Z_(k)), RSI(i,j,Z_(k)) provided by pixel p(i,j,k) forA(X_(i),Y_(j),Z_(k)), and a RCI(i,j,Z_(O)) acquired for a pixel thatimages a region of calibration surface 50 in the calibration imageacquired for range camera 20 that corresponds to regionA(X_(i),Y_(j),Z_(k)) to determine reflectance ρ(i,j,k).

In an embodiment of the disclosure, a region of calibration surface 50that corresponds to region A(X_(i),Y_(j),Z_(k)) comprises a region ofthe calibration surface that is illuminated by light from light source30 when imaged by range camera 20 to acquire the calibration image alongsubstantially a same direction along which light from light source 30illuminates region A(X_(i),Y_(j),Z_(k)) to acquire the picture image ofthe scene comprising an image of A(X_(i),Y_(j),Z_(k)). In an embodimentof the disclosure, the corresponding region is a region

(X*_(i),Y*_(j),Z_(O)) shown in FIG. 2A having coordinatesX*_(i),Y*_(j),Z_(O) in coordinate system 41 that is imaged on a pixelp(i*,j*,Z_(O)) of photosensor 22, and through which distance vectorR′(i,j,Z_(k)) passes. Coordinates X_(i),Y_(j), and Z_(k) forA(X_(i),Y_(j),Z_(k)) provided by range camera 20 from coordinates ofpixel p(i,j,k) on which the range camera images A(X_(i),Y_(j),Z_(k)),may be used to determine coordinates X*_(i),Y*_(i) in accordance withthe following equations:

X* _(i) =X _(i)+(1−(Z _(O) /Z _(k))X _(S) and Y* _(j)=(Z _(O) /Z _(k))Y_(j),  (12)

Using equations (12), the coordinates x_(i)* and y_(j)* of pixelp(i*,j*,Z_(O)) may be written

x _(i)*=−(f/Z _(O))X* _(i) and y _(j)*=−(f/Z _(O))Y* _(j).  (13)

The determined coordinates x_(i)* and y_(j)* identify a pixel p(i*,j*,Z_(O)) that provides a registered calibration irradiation,RCI(i*,j*,Z_(O)), for region

(X_(i)*,Y_(j)*,Z_(O)) corresponding to region A(X_(i),Y_(j),Z_(k)). Inaccordance with an embodiment of the disclosure, RCI(i*,j*,Z_(O)) isused to determine ρ(i,j,k) for region A(X_(i),Y_(j),Z_(k)) as describedbelow.

From equation (10),

RCI(i*,j*,Z _(O))=(τρ_(O) A _(p)α/2πf ²)Z _(O) ⁴ P _(O)(θ′,φ′,(i*,j*,Z_(O)))[R(i*,j*,Z _(O))R′(i*,*j,Z _(O))]⁻³.  (14)

where R(i*,j*,Z_(O))=[X_(i*) ²+Y_(i*) ²+Z_(O) ²]^(1/2) andR′(i*,j*,Z_(O))=[X_(i*)−X_(S))²+Y_(j*) ²+Z_(O*) ²]^(1/2). In expression(14) P_(O)(θ′,φ′,(i*,j*,Z_(O))) is the magnitude of the optical powerper unit solid angle that light source 30 transmits in the direction ofregion A(X_(i),Y_(j),Z_(k)) to illuminate A(X_(i),Y_(j),Z_(k)) withlight. Rearranging equation (11) provides an equation for reflectanceρ(i,j,k) of A(X_(i),Y_(j),Z_(k)):

ρ(i,j,k)=(τA _(p)α/2πf ²)⁻¹ Z _(k) ⁻⁴ [R(i,j,Z _(k))R′(i,j,Z_(k))]³RSI(i,j,Z _(k))/P _(O)(θ′,φ′,(i,j,Z _(k))).  (15)

Rearranging equation 10 provides an expression for optical powerP_(O)(θ′,φ′,(i,j,Z_(k))):

P _(O)(θ′,φ′,(i*,j*,Z _(O)))=(2πf ²/τρ_(O) A _(p)α)RCI(i*,j*,Z _(O))Z_(O) ⁻⁴ [R(i*,j*,Z _(O))R′(i*,*j,Z _(O))]³.  (16)

Substituting the expression for P_(O)(θ′,φ′,(i*,j*,Z_(O))) from equation(16) into equation (15) provides an equation that determines ρ(i,j,k):

ρ(i,j,k)=ρ_(O)(Z _(O) /Z _(k))⁴[RSI(i,j,Z _(k))/RCI(i*,j*,Z_(O))][(R(i,j,Z _(k))/R(i*,j*,Z _(O)))(R′(i,j,Z _(k))/R′(i*,*j,Z_(O)))]³  (17)

Equation (17) determines reflectance for region A(X_(i),Y_(j),Z_(k)) inaccordance with an embodiment of the disclosure in terms of calibrationreflectance ρ_(O), registered irradiances RSI(i,j,Z_(k)) andRCI(i*,j*,Z_(O)), and spatial coordinates provided by a range imageacquired by range camera 20.

An alternative expression for ρ(i,j,k) may be determined by normalizingcoordinates X_(i),Y_(j) and X′_(i),Y′_(j) to Z_(O) and writing equations(10) in terms of pixel coordinates x_(i) and y_(j) of a pixel p(i,j) toprovide:

RCI(i,j,Z _(O))={τ(A _(p)α/2πf ²)[1+(x _(i) /f)²+(y _(j)/f)²]^(−3/2)}ρ_(O) P _(O)(θ′,φ′,(i,j,Z _(O)))Z _(O) ⁻²[1+((x _(i) /f)−X_(S) /Z _(O))²+(y _(j) /f)²]^(−3/2).  (18)

The quantities in curly brackets are constants and independent of Z_(O)for a given pixel. Representing the quantities in brackets by a constantK(i,j), the expression (18) may be written.

RCI(i,j,Z _(O))=K(i,j)ρ_(O) P _(O)(θ′,φ′,(i,j,Z _(O)))Z _(O) ⁻²[1+((x_(i) /f)−X _(S) /Z _(O))²+(y _(j) /f)²]^(−3/2).  (19)

Similarly, equation (11) for the pixel may be written,

RSI(i,j,Z _(k))=K(i,j)ρ(i,j,k)P _(O)(θ′,φ′,(i,j,Z _(k)))Z _(k) ⁻²[1+((x_(i) /f)−X _(S) /Z _(k))²+(y _(j) /f)²]^(−3/2).  (20)

As a result, p(i,j,k) may be determined in accordance with anexpression,

ρ(i,j,k)=ρ_(O)[RSI(i,j,Z _(k),)/RCI(i,j,Z _(O),ρ_(O))](Z _(k) ² /Z _(O)²)P _(O)(θ′,φ′,(i,j,Z _(O)))/P _(O)(θ′,φ′,(i,j,Z _(k)))×[1+((x _(i)/f)−X _(S) /Z _(k))²+(y _(j) /f)²]^(3/2)/[1+((x _(i) /f)−X _(S) /Z_(O))²+(y _(j) /f)²]^(3/2).  (21)

In general, the term X_(S)/Z_(k) and X_(S)/Z_(O) are relatively smallcompared to the other terms in the expressions[1+((x_(i)/f)−X_(S)/Z_(k))²+(y_(j)/f)²] and[1+((x_(i)/f)−X_(S)/Z_(O))²+(y_(j)/f)²], and the ratio isP_(O)(θ′,φ′,(i,j,Z_(O)))/P_(O)(θ′,φ′,(i,j,Z_(k))) is a weak function ofZ_(k). The terms following the term (Z_(k) ²/Z_(O) ²) in equation (21)may therefore conveniently be expressed as a power series about thecoordinate Z_(O) optionally of the form(a_(O)(i,j,Z_(O))+a₁(i,j,Z_(O))Z_(k) ⁻¹+a₂(i,j,Z_(O))Z_(k) ⁻² - - - ) sothat

ρ(i,j,k)=ρ_(O)[RSI(i,j,Z _(k),)/RCI(i,j,Z _(O),ρ_(O))](Z _(k) ² /Z _(O)²)(a _(O)(i,j,Z _(O))+a ₁(i,j,Z _(O))Z _(k) ⁻¹+ - - - )  (22)

In an embodiment of the disclosure range camera 20 is calibrated byimaging calibration surface 50 at each of a plurality of different knowncalibration distances from the range camera to provide calibrationimages and registered calibration irradiances RCI(i,j,Z′_(O)) for aplurality of known different distances “Z′_(O)” from the range camera.Coefficients for the power series may be determined from the calibrationirradiances for each of the distances Z′_(O). Values of the calibrationirradiances RCI(i,j,Z′_(O)) and associated coefficients(a_(O)(i,j,Z_(O)), a₁(i,j,Z_(O)), . . . for each pixel may be stored ina look up table (LUT) in controller 28. The controller uses the LUTvalues for a given pixel p(i,j) to determine a reflectance ρ(i,j,k),optionally in accordance with expression (22) for a feature that rangecamera 20 images on the given pixel.

In an embodiment, distances between calibration distances Z′_(O) aredetermined so that a number of power series coefficients stored in theLUT for a pixel p(i,j) occupies a desired amount of memory in controller28, and is sufficient to provide values for ρ(i,j,k) for the pixelhaving desired accuracy in a desired controller processing time. In anembodiment, a number of power coefficients stored per pixel is less thanor equal to about 5. Optionally, the number is less than or equal to 2.

In an embodiment of the disclosure each of a plurality of range camerasof a same given type and/or make is calibrated in a calibrationprocedure in which the range camera is operated to image calibrationsurface 50 at each of a plurality of known distances to acquire anempirical LUT specific to the camera. Values for irradiances and powerseries coefficients in the acquired empirical LUTs are suitably averageto generate a generic LUT. The generic LUT may be installed in a rangecamera of the same given type and/or make for use in determiningreflectances without having to operate the range camera to acquire anempirical LUT.

In deriving equation (17)-(22) it is assumed that reflectance ofcalibration surface 50 is the same for all regions of the calibrationsurface, the calibration surface is planar, and that range camera 20illuminates the scene comprising A(X_(i),Y_(j),Z_(k)) with a sameoptical power P_(O)(θ′,φ′,(i*,j*,Z_(O))) used to acquire a calibrationimage for the range camera. Embodiments are not limited to planarcalibration surfaces having uniform reflectance ρ_(O). For example, RCIsuseable for determining a reflectance ρ(i,j,k) for features in a scenemay be determined for a cylindrical or a spherical calibration surfaceusing a modification of equations (17)-(22) that accounts for thenon-planar geometry of the calibration surface. And reflectance ofcalibration surface 50 may for example, be a known, non-constantfunction ρ_(O)(i,j,k) of coordinates X_(i),Y_(j),Z_(k) on thecalibration surface. Nor is practice of an embodiment of the disclosurelimited to illuminating a scene with optical powerP_(O)(θ′,φ′,(i*,j*,Z_(O))). Light source 30 may be operated toilluminate scenes imaged by range camera 20 at different levels ofoptical power. Equations (17)-(22) may be modified to account forilluminating a scene at an optical power different fromP_(O)(θ′,φ′,(i*,j*,Z_(O))) by multiplying the right hand side ofequation (17) and (22) by a suitable ratio between the different opticalpower and P_(O)(θ′,φ′,(i*,j*,Z_(O))).

It is noted that in the above description, the normal {circumflex over(η)}(i,j,Z_(O)) to calibration surface region

(X_(i),Y_(j),Z_(O)), and the normal {circumflex over (η)}(i,j,Z_(k)) toscene region A(X_(i),Y_(j),Z_(k)) are assumed parallel to optic axis 25of range camera 20. However, practice of an embodiment of the disclosureis not limited to determining reflectances for scene regions havingnormals parallel to optic axis 25 and for tilted scene regions amodification of expression (17) may be used to provide reflectancesρ(i,j,k) for the regions.

For example, FIG. 2B schematically shows a scenario for whichcalibration surface 50 is perpendicular to optic axis 25 and therefore{circumflex over (η)}(i,j,Z_(O)) is parallel to the optic axis, but ascene region A_(T)(X_(i),Y_(j),Z_(k)) is tilted relative to scene regionA(X_(i),Y_(j),Z_(k)) shown in FIGS. 2A and 2B. Whereas the normal{circumflex over (η)}(i,j,Z_(O)) to scene region A(X_(i),Y_(j),Z_(k)) isparallel to optic axis 25, tilted scene region A_(T)(X_(i),Y_(j),Z_(k))has a normal {circumflex over (η)}(i,j,Z_(k),T) that is tilted withrespect to optic axis 25 by optionally an angle β. As a result of thetilt, A_(T)(X_(i),Y_(j),Z_(k)) has an area A_(T)(i,j,Z_(k)) imaged onphotosensor pixel p(i,j,k) equal to

$\begin{matrix}\begin{matrix}{{A_{T}\left( {i,j,Z_{k}} \right)} = {\left\lbrack {A_{p}\left( {Z_{k}/f} \right)}^{2} \right\rbrack \left\lbrack {\cos \; {{{\theta \left( {i,j,Z_{k\;}} \right)}/{\hat{R}\left( {i,j,Z_{k}} \right)}} \cdot {\hat{\eta}\left( {i,j,Z_{k},T} \right)}}} \right\rbrack}} \\{{= {{A\left( {i,j,Z_{k}} \right)}\left\lbrack {\cos \; {{{\theta \left( {i,j,Z_{k}} \right)}/{\hat{R}\left( {i,j,Z_{k}} \right)}} \cdot {\hat{\eta}\left( {i,j,Z_{k},T} \right)}}} \right\rbrack}},}\end{matrix} & (23)\end{matrix}$

where [A_(p)(Z_(k)/f)²] is the area A(i,j,Z_(k)) of non-tilted sceneregion A(X_(i),Y_(j),Z_(k)) determined similarly to determining

(i,j,Z_(O)) in accordance with equation (1).

In addition, in an equation for total irradianceTIR(A_(T)(X_(i),Y_(j),Z_(k))) of tilted scene regionA_(T)(X_(i),Y_(j),Z_(k)) similar to equation (2) for TIR(

(X_(i),Y_(j),Z_(O))), it may not be appropriate, as provided for TIR(

(X_(i),Y_(j),Z_(O))) by equation (3), to replace {circumflex over(η)}(i,j,Z_(k),T)·P_(O)(θ′,φ′,(i,j,Z_(k))) withP_(O)(θ′,φ′,(i,j,Z_(k)))[Z_(k)/R′(i,j,Z_(k))]. In equation (3),{circumflex over (η)}(i,j,Z_(O))·P_(O)(θ′,φ′,(i,j,Z_(O))) is replaced byP_(O)(θ′,φ′,(i,j,Z_(O)))[Z_(O)/R′(i,j,Z_(O))] under the assumption thatnormal {circumflex over (η)}(i,j,Z_(O)) to

(X_(i),Y_(j),Z_(O)) is parallel to optic axis 25 and as a result{circumflex over (η)}(i,j,Z_(O))·{circumflex over(P)}_(O)(θ′,φ′,(i,j,Z_(O)))=cos θ′(i,j,Z_(O))=[Z_(O)/R′(i,j,Z_(O))].Irradiance RSI(i,j,Z_(k)) of A(X_(i),Y_(j),Z_(k)) given by expression(11) is similarly dependent on the assumption that normal {circumflexover (η)}(i,j,Z_(k)) to A(X_(i),Y_(j),Z_(k)) is parallel to optic axis25 and therefore that {circumflex over (η)}(i,j,Z_(k))·{circumflex over(P)}_(O)(θ′,φ′,(i,j,Z_(k)))=cos θ′(i,j,Z_(k))=[Z_(k)/R′(i,j,Z_(k))].

Since the assumption that {circumflex over (η)}(i,j,Z_(k)) is parallelto optic axis 25 and its consequence that cosθ′(i,j,Z_(O))=[Z_(O)/R′(i,j,Z_(O))] do not obtain for {circumflex over(η)}(i,j,Z_(k),T), equation (17) may not provide a satisfactory valuefor ρ(i,j,k) for tilted scene region A_(T)(X_(i),Y_(j),Z_(k)).

Therefore, in an embodiment of the disclosure, for a tilted scene regionsuch as A_(T)(X_(i),Y_(j),Z_(k)) equation (17) may be adjusted for tiltby multiplying the right hand side of the equation by a “tilt factor”(TF) defined by an expression:

TF=[Z_(k) /R′(i,j,Z _(k))cos θ(i,j,Z _(k))][{circumflex over (R)}(i,j,Z_(k))·{circumflex over (η)}(i,j,k,T)/{circumflex over (R)}′(i,j,Z_(k))·{circumflex over (η)}(i,j,k,T)].  (24)

And if reflectance for a tilted scene region is represented byρ_(T)(i,j,k), then ρ_(T)(i,j,k)=TFρ(i,j,k), which if ρ(i,j,k) isdetermined in accordance with equation (17), equals

TFρ_(O)(Z _(O) /Z _(k))⁴[RSI(i,j,Z _(k))/RCI(i*,j*,Z _(O))][(R(i,j,Z_(k))/R(i*,j*,Z _(O)))(R′(i,j,Z _(k))/R′(i*,*j,Z _(O)))]³.  (25)

The expression for TF assumes that registered calibration irradianceRCI(i,j,Z_(O)) is determined for a planar calibration surface 50 asshown in FIG. 1 and defined by equation (10), and adjusts equation (17)or (22) to account for a difference in direction between {circumflexover (η)}(i,j,Z_(O)) and {circumflex over (η)}(i,j,k,T) in determiningtotal irradiance of A_(T)(i,j,Z_(k)) by light from light source 42. Inparticular it is noted that total irradiance TIR(A(X_(i),Y_(j),Z_(k)))of a scene region A(X_(i),Y_(j),Z_(k)) having normal parallel to opticaxis 25 may be written, similarly to equations (2) and (5):

$\begin{matrix}\begin{matrix}{{{TIR}\left( {A\left( {X_{i},Y_{i},Z_{k}} \right)} \right)} = {{A\left( {i,j,Z_{o}} \right)}{{\hat{\eta}\left( {i,j,Z_{k}} \right)} \cdot}}} \\{{{\left. {{{{P_{o}\left( \theta ’ \right.},\phi}’},\left( {i,j,Z_{k}} \right)} \right)/R}’}\left( {i,j,Z_{k}} \right)^{2}} \\{\left. {{\left. {{{{= {\left( {A_{p}/f^{2}} \right){P_{o}\left( \theta ’ \right.}}},\phi}’},\left( {i,j,Z_{k}} \right)} \right)\left\lbrack {Z_{k}/R}’ \right.}\left( {i,j,Z_{k}} \right)^{3}} \right\rbrack.}\end{matrix} & (26)\end{matrix}$

Total irradiance of tilted scene region A_(T)(X_(i),Y_(j),Z_(k)) bylight from light source 42 may similarly be written,

$\begin{matrix}\begin{matrix}{{{TIR}\left( {A_{T}\left( {X_{i},Y_{j},Z_{k}} \right)} \right)} = {{A_{T}\left( {i,j,Z_{k}} \right)}{{\hat{\eta}\left( {i,j,Z_{k},T} \right)} \cdot}}} \\{{{\left. {{{{P_{o}\left( \theta ’ \right.},\phi}’},\left( {i,j,Z_{k}} \right)} \right)/R}’}\left( {i,j,Z_{k}} \right)^{2}} \\\left. {{{{= {{A_{T}\left( {i,j,Z_{k}} \right)}{P_{o}\left( \theta ’ \right.}}},\phi}’},\left( {i,j,Z_{k}} \right)} \right) \\{{{\left. {\left( {{\hat{\eta}\left( {i,j,Z_{k},T} \right)} \cdot \hat{R}}’ \right.\left( {i,j,Z_{k}} \right)} \right)/R}’}\left( {i,j,Z_{k}} \right)^{2}} \\\left. {{{{= {{A_{T}\left( {i,j,Z_{k}} \right)}{P_{o}\left( \theta ’ \right.}}},\phi}’},\left( {i,j,Z_{k}} \right)} \right) \\\left. {\left( {{\hat{\eta}\left( {i,j,Z_{k},T} \right)} \cdot \hat{R}}’ \right.\left( {i,j,Z_{k}} \right)} \right) \\{\left. {{\left. {\left( R’ \right.{\left( {i,j,Z_{k}} \right)/Z_{k}}} \right)\left\lbrack {Z_{k}/R}’ \right.}\left( {i,j,Z_{k}} \right)^{3}} \right\rbrack.}\end{matrix} & (27)\end{matrix}$

Using expression (23) for A_(T)(i,j,Z_(k)) in equation (27) results in:

$\begin{matrix}\begin{matrix}{{{TIR}\left( {A_{T}\left( {X_{i},Y_{j},Z_{k}} \right)} \right)} = {\left\lbrack {\cos \; {{{\theta \left( {i,j,Z_{k}} \right)}/{\hat{R}\left( {i,j,Z_{k}} \right)}} \cdot {\hat{\eta}\left( {i,j,Z_{k},T} \right)}}} \right\rbrack \cdot}} \\{\left. {\left. {\left( {{\hat{\eta}\left( {i,j,Z_{k},T} \right)} \cdot R}’ \right.\left( {i,j,Z_{k}} \right)} \right)\left( R’ \right.{\left( {i,j,Z_{k}} \right)/Z_{k}}} \right) \times} \\\left. {{{{\left\lbrack {A_{p}\left( {Z_{k}/f} \right)}^{2} \right\rbrack {P_{o}\left( \theta ’ \right.}},\phi}’},\left( {i,j,Z_{k}} \right)} \right) \\\left. {\left\lbrack {Z_{k}/R}’ \right.\left( {i,j,Z_{k}} \right)^{3}} \right\rbrack \\{= \left\lbrack {\cos \; {{{\theta \left( {i,j,Z_{k}} \right)}/{\hat{R}\left( {i,j,Z_{k}} \right)}} \cdot {\hat{\eta}\left( {i,j,Z_{k},T} \right)}}} \right\rbrack} \\{\left. {\left( {{\hat{\eta}\left( {i,j,Z_{k},T} \right)} \cdot R}’ \right.{\left( {i,j,Z_{k}} \right)/Z_{k}}} \right) \times} \\{{{TIR}\left( {A\left( {X_{i},Y_{j},Z_{k}} \right)} \right.}} \\\left. {= {\cos \; {\theta \left( {i,j,Z_{k}} \right)}\left( R’ \right.{\left( {i,j,Z_{k}} \right)/Z_{k}}}} \right) \\{{\left\lbrack \frac{\left. {{{\left( {{\hat{\eta}\left( {i,j,Z_{k}} \right)},T} \right) \cdot R}’}\left( {i,j,Z_{k}} \right)} \right)}{\left( {{\hat{R}\left( {i,j,Z_{k}} \right)} \cdot {\hat{\eta}\left( {i,j,Z_{k},T} \right)}} \right.} \right\rbrack \times}} \\{{{TIR}\left( {A\left( {X_{i},Y_{j},Z_{k}} \right)} \right.}} \\{= {{TF}^{- 1}{{TIR}\left( {{A\left( {X_{i},Y_{j},Z_{k}} \right)}.} \right.}}}\end{matrix} & (28)\end{matrix}$

Expression (28) defines TF for planar calibration surface 50 and atilted scene region TIR(A_(T) (X_(i),Y_(j),Z_(k))).

Equations (23)-(28) express variables and reflectance of a tiltedsurface region A_(T)(i,j,Z_(k)) of a scene imaged by range camera 20 asa function of a unit normal, {circumflex over (η)}(i,j,Z_(k),T) of theregion, which is not necessarily parallel to optic axis 25. In anembodiment of the disclosure, range camera 20 acquires 3D spatialcoordinates, X_(i), Y_(j), and Z_(k), for points in a scene that definea range image of the scene and thereby 3D surfaces for features in thescene. In an embodiment of the disclosure, range camera 20 processes the3D spatial coordinates to determine gradients for regions of thesurfaces and therefrom normals {circumflex over (η)}(i,j,Z_(k),T) to thesurface regions. The normals are optionally used in accordance withequations (23)-(28) or equations similar to equations (23)-(28) todetermine reflectances for the surface regions. It is noted that whereasequations (23)-(28) relax a constraint that {circumflex over(η)}(i,j,Z_(k),T) is parallel to optic axis 50, the equations areconstrained by an assumption that intensity of reflected light per unitsolid angle from an illuminated surface is independent of angle ofreflection relative to normals {circumflex over (η)}(i,j,Z_(k)) and{circumflex over (η)}(i,j,Z_(k),T). In an embodiment of the disclosurethe constraint may be relaxed to determine RCI(i,j,Z_(O)) and it may beassumed for example that intensity of reflected light decreasesproportional to the cosine of an angle that direction of propagation ofthe reflected light makes relative to normal {circumflex over(η)}(i,j,Z_(k)) and/or normal {circumflex over (η)}(i,j,Z_(k),T)

FIG. 3 schematically shows a user 180 interacting with a human computerinterface (HCl) 200 optionally comprising a video screen 202 forproviding visual information to the user and a range camera 20 having anoptic axis 25 (FIG. 1). Range camera 20 images the user and usergestures to communicate information from the user to a computer (notshown), and operates in accordance with an embodiment of the disclosureto determine reflectance of regions of the facial skin of user 180 tocharacterize the user and user gestures. Controller 28 of camera 20optionally comprises a calibration image having RCIs for pixels 23 inphotosensor 22 determined in accordance with an embodiment of thedisclosure. Optionally, the RCIs are determined as described withreference to FIG. 1. Controller 28 controls range camera 20 to acquirerange images and picture images of user 180 and, optionally, determinesskin reflectance for facial skin regions of user 180 responsive to therange and picture images of the user, and RCIs comprised in thecalibration image.

Controller 28 may determine skin reflectance for regions of facial skinof user 180 in accordance with equation (17) or equation (22) under theassumption that normals to regions of the user's facial skin aresubstantially parallel to optic axis 25 of the range camera. Controller28 may determine skin reflectance for regions of facial skin of user 180in accordance with equations (23-28) after determining normals{circumflex over (η)}(i,j,Z_(k),T) to the user's facial skin regions.Optionally, the normals are determined by calculating gradients tosurfaces of the user's facial skin responsive to 3D spatial coordinatesprovided by the range images. FIG. 3 schematically shows facial skinregions, represented by small rectangles 182 of user 180 and normals{circumflex over (η)}(i,j,Z_(k),T) schematically represented by arrows184 to the facial skin regions.

In an embodiment of the disclosure range camera 20 illuminates the faceof user 180 with IR light and monitors changes in IR reflectance of theuser's facial skin region to determine the user's heartbeat and/ortemperature. Optionally controller 28 uses changes in reflectance toprovide information characterizing the user's emotional state. Forexample to determine if the user is blushing or angry.

There is therefore provided in accordance with an embodiment of thedisclosure a range camera operable to determine distances to andreflectance for features of a scene that the range camera images, therange camera comprising: a light source operable to transmit light toilluminate a scene that the range camera images; a photosensor havinglight sensitive pixels; optics having an optical axis and optical centerconfigured to receive light reflected from the transmitted light byfeatures in the scene and image the reflected light on a plurality ofthe pixels, each of which plurality of pixels registers reflected lightfrom a feature of the scene that the optics images on the pixel toprovide a registered scene irradiance (RSI) for the feature; acalibration image comprising a registered calibration irradiance (RCI)for each of the light sensitive pixels that provides a measure ofirradiance that the pixel registers responsive to light reflected fromlight transmitted by the light source by a region of a calibrationsurface having known shape, reflectance, and location that the rangecamera images on the pixel; and a controller operable to determine areflectance for a feature of the scene imaged on a first pixel of thephotosensor comprised in the plurality of pixels responsive to the RSIfor the feature and an RCI for a region of the calibration surfaceimaged on a second pixel of the photosensor corresponding to the featureimaged on the first pixel.

In an embodiment, the feature imaged on the first pixel and thecorresponding region of the calibration surface imaged on the secondpixel are imaged on their respective pixels with light they reflect fromlight transmitted along a same direction by the light source. Thedetermined reflectance for the feature imaged on the first pixel may bea function of a ratio RSI/RCI times reflectance ρ_(O) of thecorresponding region of the calibration surface.

In an embodiment, the range camera determines a distance vector R forthe feature that defines a location of the feature in a field of view(FOV) of the camera relative to the optics and a distance vector R′ thatdefines a location for the feature imaged on the first pixel in the FOVrelative to the light source and uses R and R′ to determine a locationon the calibration surface of the corresponding region of thecalibration surface. Optionally, the range camera determines a distancevector R_(O) for the corresponding region of the calibration surface inthe FOV of the range camera relative to the optics, and a distancevector R_(O)′ that defines a location of the corresponding region in theFOV relative to a location of the light source responsive to thedetermined location of the corresponding region of the calibrationsurface.

In an embodiment, the reflectance for the feature imaged on the firstpixel is proportional to ρ_(O)(RSI/RCI)[(|R|/|R_(O)|)(|R′|/|R′_(O)|)]³where ρ_(O) is reflectance of the corresponding region of thecalibration surface.

Optionally, the calibration surface is oriented perpendicular to anoptical axis of the camera at a fixed distance Z_(O) from the opticalcenter along the optical axis and the feature imaged on the first pixelis located at a distance Z from the optical center along the opticalaxis.

In an embodiment, the reflectance for the feature imaged on the firstpixel is proportional toρ_(O)(Z_(O)/Z)⁴(RSI/RCI)[(|R|/|R_(O)|)(|R′|/|R′_(O)|)]³ where ρ_(O) isreflectance of the corresponding region of the calibration surface.

In an embodiment of the disclosure, the controller is operable todetermine a tilt factor TF for the feature imaged on the first pixel anduse the tilt factor to determine the reflectance for the feature,wherein TF=[Z/(|R′|({circumflex over (R)}·{circumflex over(z)}))][({circumflex over (R)}·{circumflex over (η)})/({circumflex over(R)}′·{circumflex over (η)})] and {circumflex over (η)} is a unit normalto a surface of the feature imaged on the first pixel, “Z” is distanceof the feature from the optical center along the optical axis, and{circumflex over (R)} and {circumflex over (R)}′ are unit vectorspointing from the feature to the optical center and the light sourcerespectively. Optionally, the reflectance for the feature imaged on thefirst pixel is proportional toTFρ_(O)(Z_(O)/Z)⁴(RSI/RCI)[(|R|/|R_(O)|)(|R′|/|R′_(O)|)]³ where ρ_(O) isa known reflectance of the region imaged on the second pixel. Thecontroller may control the range camera to acquire a range image of thescene and determines {circumflex over (η)} responsive to a gradient of asurface of the feature defined by the range image.

In an embodiment of the disclosure, the reflectance of the calibrationsurface is substantially the same for all regions of the calibrationsurface having an RCI comprised in the calibration image. In anembodiment of the disclosure, the calibration surface is planar.

There is further provided in accordance with an embodiment of thedisclosure a method of determining reflectance for a feature of a scene,the method comprising: using an active lighting range camera to acquirea range image and a picture image of the scene responsive to lighttransmitted by the range camera to illuminate the scene that isreflected by features in the scene back to the range camera; determininga location of the feature in the FOV responsive to the range image;determining registered irradiance of the range camera by light reflectedfrom the transmitted light by the feature responsive to the pictureimage; determining a corresponding region of a calibration surfaceimaged in a calibration image acquired by the range camera; determiningregistered irradiance of the range camera by light reflected from thetransmitted light by the corresponding region responsive to thecalibration image; and using the determined registered irradiances fromthe feature and the corresponding calibration region and knownreflectance of the calibration surface to determine the reflectance ofthe feature.

Optionally, determining the corresponding region of the calibrationsurface comprises determining a region of the calibration surface forwhich light from the range camera that illuminates the region to acquirethe calibration image is transmitted along a same direction that therange camera illuminates the feature to acquire the picture image.Determining the reflectance of the feature may comprise multiplying theknown reflectance by a ratio equal to the registered irradiance from thefeature divided by the registered irradiance from the correspondingregion.

The method may comprise determining a unit normal {circumflex over (η)}to a surface of the feature responsive to the range image and using theunit normal to determine the reflectance. Optionally, using the normalcomprises determining a first distance vector from an optical center ofthe range camera to the feature responsive to the range image anddetermining a first scalar product of the normal with the first distancevector. Using the normal may comprise determining a second distancevector from a location from which the range camera transmits light toilluminate the scene to the feature responsive to the range image anddetermining a second scalar product of the normal with the seconddistance vector. Optionally, determining the reflectance comprisesdetermining a ratio between the first scalar product and the secondscalar product.

In the description and claims of the present application, each of theverbs, “comprise” “include” and “have”, and conjugates thereof, are usedto indicate that the object or objects of the verb are not necessarily acomplete listing of components, elements or parts of the subject orsubjects of the verb.

Descriptions of embodiments of the disclosure in the present applicationare provided by way of example and are not intended to limit the scopeof the disclosure. The described embodiments comprise differentfeatures, not all of which are required in all embodiments. Someembodiments utilize only some of the features or possible combinationsof the features. Variations of embodiments of the disclosure that aredescribed, and embodiments comprising different combinations of featuresnoted in the described embodiments, will occur to persons of the art.The scope of the invention is limited only by the claims

1. A range camera operable to determine distances to and reflectance forfeatures of a scene that the range camera images, the range cameracomprising: a light source operable to transmit light to illuminate ascene that the range camera images; a photosensor having light sensitivepixels; optics having an optical axis and optical center configured toreceive light reflected from the transmitted light by features in thescene and image the reflected light on a plurality of the pixels, eachof which plurality of pixels registers reflected light from a feature ofthe scene that the optics images on the pixel to provide a registeredscene irradiance (RSI) for the feature; a calibration image comprising aregistered calibration irradiance (RCI) for each of the light sensitivepixels that provides a measure of irradiance that the pixel registersresponsive to light reflected from light transmitted by the light sourceby a region of a calibration surface having known shape, reflectance,and location that the range camera images on the pixel; and a controlleroperable to determine a reflectance for a feature of the scene imaged ona first pixel of the photosensor comprised in the plurality of pixelsresponsive to the RSI for the feature and an RCI for a region of thecalibration surface imaged on a second pixel of the photosensorcorresponding to the feature imaged on the first pixel.
 2. The rangecamera according to claim 1 wherein the feature imaged on the firstpixel and the corresponding region of the calibration surface imaged onthe second pixel are imaged on their respective pixels with light theyreflect from light transmitted along a same direction by the lightsource.
 3. The range camera according to claim 1 wherein the determinedreflectance for the feature imaged on the first pixel is a function of aratio RSI/RCI times reflectance ρ_(O) of the corresponding region of thecalibration surface.
 4. The range camera according to claim 1 whereinthe range camera determines a distance vector R for the feature thatdefines a location of the feature in a field of view (FOV) of the camerarelative to the optics and a distance vector R′ that defines a locationfor the feature imaged on the first pixel in the FOV relative to thelight source and uses R and R′ to determine a location on thecalibration surface of the corresponding region of the calibrationsurface.
 5. The range camera according to claim 4 wherein the rangecamera determines a distance vector R_(O) for the corresponding regionof the calibration surface in the FOV of the range camera relative tothe optics, and a distance vector R_(O)′ that defines a location of thecorresponding region in the FOV relative to a location of the lightsource responsive to the determined location of the corresponding regionof the calibration surface.
 6. The range camera according to claim 5wherein the reflectance for the feature imaged on the first pixel isproportional to ρ_(O)(RSI/RCI)[(|R|/|R_(O)|)(|R|/|R′_(O)|)]³ where ρ_(O)is reflectance of the corresponding region of the calibration surface.7. The range camera according to claim 6 wherein the calibration surfaceis oriented perpendicular to an optical axis of the camera at a fixeddistance Z_(O) from the optical center along the optical axis and thefeature imaged on the first pixel is located at a distance Z from theoptical center along the optical axis.
 8. The range camera according toclaim 7 wherein the reflectance for the feature imaged on the firstpixel is proportional toρ_(O)(Z_(O)/Z)⁴(RSI/RCI)[(|R|/|R_(O)|)(|R′|/|R′_(O)|)]³ where ρ_(O) isreflectance of the corresponding region of the calibration surface. 9.The range camera according to claim 1 wherein the controller is operableto determine a tilt factor TF for the feature imaged on the first pixeland use the tilt factor to determine the reflectance for the feature,wherein TF=[Z/(|R′|({circumflex over (R)}·{circumflex over(Z)}))][({circumflex over (R)}·{circumflex over (η)})/({circumflex over(R)}′·{circumflex over (η)})] and {circumflex over (η)} is a unit normalto a surface of the feature imaged on the first pixel, “Z” is distanceof the feature from the optical center along the optical axis, and{circumflex over (R)} and {circumflex over (R)}′ are unit vectorspointing from the feature to the optical center and the light sourcerespectively.
 10. The range camera according to claim 9 wherein thereflectance for the feature imaged on the first pixel is proportional toTFρ_(O)(Z_(O)/Z)⁴(RSI/RCI)[(|R|/|R_(O)|)(|R′|/|R′_(O)|)]³ where ρ_(O) isa known reflectance of the region imaged on the second pixel.
 11. Therange camera according to claim 9 wherein the controller controls therange camera to acquire a range image of the scene and determines{circumflex over (η)} responsive to a gradient of a surface of thefeature defined by the range image.
 12. The range camera according toclaim 1 wherein the reflectance of the calibration surface issubstantially the same for all regions of the calibration surface havingan RCI comprised in the calibration image.
 13. The range cameraaccording to claim 1 wherein the calibration surface is planar.
 14. Amethod of determining reflectance for a feature of a scene, the methodcomprising: using an active lighting range camera to acquire a rangeimage and a picture image of the scene responsive to light transmittedby the range camera to illuminate the scene that is reflected byfeatures in the scene back to the range camera; determining a locationof the feature in the FOV responsive to the range image; determiningregistered irradiance of the range camera by light reflected from thetransmitted light by the feature responsive to the picture image;determining a corresponding region of a calibration surface imaged in acalibration image acquired by the range camera; determining registeredirradiance of the range camera by light reflected from the transmittedlight by the corresponding region responsive to the calibration image;and using the determined registered irradiances from the feature and thecorresponding calibration region and known reflectance of thecalibration surface to determine the reflectance of the feature.
 15. Themethod according to claim 14 wherein determining the correspondingregion of the calibration surface comprises determining a region of thecalibration surface for which light from the range camera thatilluminates the region to acquire the calibration image is transmittedalong a same direction that the range camera illuminates the feature toacquire the picture image.
 16. The method according to claim 14 whereindetermining the reflectance of the feature comprises multiplying theknown reflectance by a ratio equal to the registered irradiance from thefeature divided by the registered irradiance from the correspondingregion.
 17. The method according to claim 16 and comprising determininga unit normal {circumflex over (η)} to a surface of the featureresponsive to the range image and using the unit normal to determine thereflectance.
 18. The method according to claim 17 wherein using thenormal comprises determining a first distance vector from an opticalcenter of the range camera to the feature responsive to the range imageand determining a first scalar product of the normal with the firstdistance vector.
 19. The method according to claim 18 wherein using thenormal comprises determining a second distance vector from a locationfrom which the range camera transmits light to illuminate the scene tothe feature responsive to the range image and determining a secondscalar product of the normal with the second distance vector.
 20. Themethod according to claim 19 wherein determining the reflectancecomprises determining a ratio between the first scalar product and thesecond scalar product.
 21. A range camera operable to determinedistances to and reflectance for features of a scene that the rangecamera images, the range camera comprising: a light source operable totransmit light to illuminate a scene that the range camera images; aphotosensor having a plurality of light sensitive pixels; a look uptable (LUT) having for each pixel of the plurality of pixels a set ofvalues for a first feature having known reflectance that is imaged onthe pixel from a known first distance from the camera and illuminated bylight at known intensity from the light source, the set of valuescomprising a registered calibration irradiance (RCI) for light reflectedby the first feature from the illuminating light; and a controller thatdetermines a reflectance of a second feature imaged on the pixel from asecond distance responsive to square of the ratio of the first distancedivided by the second distance, the RCI, and a registered sceneirradiance (RSI) that the pixel provides for light from the light sourcethat is reflected by the second feature and imaged on the pixel.
 22. Therange camera according to claim 21 wherein the LUT comprises at leastone coefficient of a power series that provides an expected registeredirradiance that the pixel generates for light from the light sourcereflected by the first feature for a displacement of the first featurefrom the first distance.
 23. The range camera according to claim 22wherein the controller determines the reflectance responsive to acoefficient of the at least one coefficient.